Impacts of User Modeling on Personalization of Information Retrieval: An Evaluation with Human Intelligence Analysts

User modeling is the key element in assisting intelligence analysts to meet the challenge of gathering relevant information from the massive amounts of available data. We have developed a dynamic user model to predict the analyst’s intent and help the information retrieval application better serve the analyst’s information needs. In order to justify the effectiveness of our user modeling approach, we have conducted a user evaluation study with actual end user, three working intelligence analysts, and compared our user model enhanced information retrieval system with a commercial off-the-shelf system, the Verity Query Language. We describe our experimental setup and the specific metrics essential to evaluate user modeling for information retrieval. The results show that our user modeling approach tracked individual’s interests, adapted to their individual searching strategies, and helped retrieve more relevant documents than the Verity Query Language system.